Simulated-driving-environment generation

ABSTRACT

A method may include obtaining HD mapping data including multiple data topics with one or more distinct instances. The method may include creating multiple meshes from corresponding distinct instances. The method may include defining multiple textures for each data topic. The method may include selecting a first data topic and a first distinct instance of the first data topic. The method may include selecting a first texture for the first data topic and applying the first texture to a first mesh corresponding with the first distinct instance. The method may include selecting a second distinct instance of the first data topic. The method may include selecting a second texture and applying the second texture to a second mesh corresponding with the second distinct instance. The method may also include combining the first mesh and the first texture with the second mesh and the second texture to generate a combined mesh.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/424,468, filed May 28, 2019, titled SIMULATED-DRIVING-ENVIRONMENTGENERATION, which is incorporated herein by reference in its entirety.

FIELD

The embodiments discussed in the present disclosure are related tosimulated-driving-environment generation.

BACKGROUND

A simulated environment may be used to test software. For example, asimulated driving environment may be used to test software of anautonomous vehicle.

The subject matter claimed in the present disclosure is not limited toembodiments that solve any disadvantages or that operate only inenvironments such as those described above. Rather, this background isonly provided to illustrate one example technology area where someembodiments described in the present disclosure may be practiced.

SUMMARY

A method may include obtaining HD mapping data including multiple datatopics with one or more distinct instances. The method may includecreating multiple meshes from corresponding distinct instances. Themethod may include defining multiple textures for each data topic. Themethod may include selecting a first data topic and a first distinctinstance of the first data topic. The method may include selecting afirst texture for the first data topic and applying the first texture toa first mesh corresponding with the first distinct instance. The methodmay include selecting a second distinct instance of the first datatopic. The method may include selecting a second texture and applyingthe second texture to a second mesh corresponding with the seconddistinct instance. The method may also include combining the first meshand the first texture with the second mesh and the second texture togenerate a combined mesh.

The objects and advantages of the embodiments will be realized andachieved at least by the elements, features, and combinationsparticularly pointed out in the claims.

Both the foregoing general description and the following detaileddescription are given as examples and are explanatory and are notrestrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 is a diagram representing an example environment related togenerating a simulated driving environment;

FIG. 2 is a diagram representing another example environment related togenerating a simulation environment;

FIG. 3 illustrates an example point cloud;

FIG. 4 illustrates example meshes;

FIG. 5 illustrates a block diagram of an example computing system;

FIG. 6 is a flowchart of an example method of generating a simulationenvironment; and

FIG. 7 is a flowchart of another example method of generating asimulation environment,

all arranged according to at least one embodiment described in thepresent disclosure.

DESCRIPTION OF EMBODIMENTS

An autonomous-vehicle software testbed may be used to testautonomous-vehicle software. The autonomous-vehicle software testbed maysimulate a driving environment to test the autonomous-vehicle software(e.g., to determine how the autonomous-vehicle software would perform invarious driving situations). For example, the autonomous-vehiclesoftware testbed may simulate a three dimensional (3-D) geographicalenvironment and provide inputs to the autonomous-vehicle software thatare analogous to inputs that the autonomous-vehicle software may receivein real-world driving conditions if the autonomous-vehicle software wererunning on an autonomous vehicle navigating within the actualgeographical environment.

For instance, the autonomous-vehicle software testbed may simulate a 3-Dcity, including roads and various objects included in the 3-D city.Additionally, the corresponding autonomous vehicle may include a cameraand the autonomous-vehicle software may use images captured by thecamera to navigate the 3-D city, In these or other embodiments, theautonomous-vehicle software testbed may provide images and/or videoanalogous to what the camera on the autonomous vehicle might capture ifthe autonomous vehicle were driving in the simulated 3-D city to testhow the autonomous-vehicle software may respond to such images.

Some embodiments described in the present disclosure relate to methodsand systems of generating a simulated 3-D geographical environment thatmay be used with an autonomous-vehicle software testbed. In someembodiments, the 3-D geographical environment may be generated based onreal-world map data.

For example, in some embodiments, operations of the present disclosuremay include obtaining map data of a geographic area that may includeinformation about roads and areas between roads.

In some cases, some of the information may be more detailed than theinformation that is likely to be used by the autonomous-vehiclesoftware. For example, the map data may include a detailed map of thetop of a building.

In some embodiments, the operations may include filtering the obtainedmap data and using the filtered map data to generate the simulated 3-Dgeographical environment. In these or other embodiments, the filteringmay be based on sensor information that may be used by theautonomous-vehicle software. For example, if an autonomous vehicleincludes cameras and Light Detection and Ranging (LIDAR) sensors thatare pointed horizontally and below horizontal, the operations of thepresent disclosure may include filtering out map data that includesdetails of objects above a certain height above the road because suchdetails may not be detected by the LIDAR sensors and cameras andconsequently may not be used by the autonomous-vehicle software. Areasabove a certain height may be outside of the sensor field of view, thuslittle or no data may be obtained above the height.

In some embodiments, the operations of the present disclosure mayinclude generating a 3-D environmental representation of the geographicarea based on the filtered map data. The 3-D environmentalrepresentation may be used as part of a 3-D geographical environmente.g., in an autonomous-vehicle software testbed. In some embodiments,the 3-D environmental representation may include a 3-D mesh generatedfrom HD map data, such as, for example, a 3-D mesh generated from apoint cloud. In these and other embodiments, the point cloud may includea collection of points to represent objects in space. For example, apoint cloud may include data generated from a LIDAR sensor. In these andother embodiments, the 3-D mesh may include multiple polygons createdusing the points in the point cloud as reference points to define thepolygons.

The filtered map data may include less data than the unfiltered mapdata. As such, it may take less processing time to generate the 3-Denvironmental representation based on the filtered map data than togenerate a 3-D environmental representation based on the unfiltered mapdata. Additionally, because the filtering of the map data may be basedon how the autonomous-vehicle software may use inputs based on the mapdata, a 3-D environmental representation based on filtered map data maybe just as useful for testing autonomous-vehicle software as a 3-Denvironmental representation based on unfiltered data. Thus, theoperations of the present disclosure may constitute an improvement inthe generation of a 3-D environmental representation because the 3-Denvironmental representation based on the filtered map data may havebeen generated faster and/or more efficiently. Therefore, filtering mapdata for generation of a 3-D environmental representation may constitutean improvement in computer-related technology (e.g., 3-D environmentalrepresentation generation and/or autonomous-vehicle software testing).

Additionally, filtering map data may have a practical application atleast in the field of 3-D modeling and/or autonomous-vehicle softwaretesting. For example, filtering map data may be applied to the field of3-D modeling to reduce unnecessary information in 3-D models which mayresult in 3-D model generation that is faster and/or more efficientwhile maintaining utility of the resulting 3-D models. Additionally oralternatively, 3-D models without unnecessary information may requireless storage space and may be able to be rendered faster and/or moreefficiently.

In addition, in some embodiments, the map data may include highdefinition (HD) mapping data. The HD mapping data may include data in apoint cloud and may be separated into different data topics. In somecases, 3-D environment representation generation may include generatingone or more meshes based on the point cloud.

In these or other embodiments, the point cloud data that corresponds todifferent data topics may be organized as individual meshes that mayeach represent a corresponding data topic. For example, in someembodiments, the 3-D environment representation generation operationsmay include separately processing each data topic of the HD mapping datato generate meshes for each data topic. In some embodiments, multipletextures may be created for each data topic. In these and otherembodiments, the multiple textures may enable the resulting simulationenvironment to more realistically reflect actual environments in whichan autonomous vehicle may operate. In some embodiments, the HD mappingdata may be filtered prior to being processed to generate the meshes.

In some embodiments, the HD mapping data may include LIDAR data fromLIDAR sensors. In these and other embodiments, the LIDAR sensors mayobtain many points arranged in a point cloud. In these and otherembodiments, the point cloud may include a collection of points torepresent an object in space. For example, each point in a point cloudmay include both a positional component and an intensity component. Forexample, each point in the point cloud may be associated with x-, y-,and z-coordinates and with an intensity. In these and other embodiments,the intensity of a point may be associated with a particular data topic,or type of object in the environment. For example, lane lines may beassociated with a different intensity than curb edges or trees. In someembodiments, the operations of the present disclosure may includegenerating one or more meshes with one or more textures based on the HDmapping data or based on filtered HD mapping data from LIDAR sensors.

In some embodiments, the operations of the present disclosure mayinclude generating the 3-D environmental representation of thegeographic area by combining a variety of meshes together with texturesfor each mesh to create an overall mesh.

Separately generating meshes for each data topic and generating multipletextures for each data topic may result in more robust 3-D simulationenvironments for autonomous vehicles. For example, separately generatingmeshes for each data topic may include less data than generating meshesfor an entire point cloud at once. For example, each data topic may begenerated using different filtering of the associated data points fromthe point cloud, which may allow the data topics to retain accuracywithout including extraneous information. As such, it may take lessprocessing time to generate the combined mesh for the point cloud whenseparately processing each data topic than when processing the pointcloud as a whole at once. Additionally, because multiple differenttextures may be applied to instances of a single data topic, a singlesimulation environment may present an autonomous vehicle with differentdepictions of a singular element, such as lane lines, which may improvethe ability of the simulation environment to test the autonomousvehicle. In addition, applying textures to the meshes may improve thespeed of generating a simulation environment and/or enable thesimulation environment to be modified for different scenarios. Thus, theoperations of the present disclosure may constitute an improvement inthe generation of a 3-D environmental representation because the 3-Dmeshes may have been generated faster and/or more efficiently.Therefore, the meshes and/or textures used for generation of a 3-Denvironmental representation may constitute an improvement incomputer-related technology (e.g., 3-D environmental representationgeneration and/or autonomous-vehicle software testing).

Additionally, separately processing each data topic may have a practicalapplication at least in the field of 3-D modeling and/orautonomous-vehicle software testing. For example, separately processingeach data topic may be applied to the field of 3-D modeling to optimizethe filtering of data points in a data cloud to reduce unnecessaryinformation in 3-D models which may result in 3-D model generation thatis faster and/or more efficient while maintaining utility of theresulting 3-D models. Additionally or alternatively, 3-D models withoutunnecessary information may require less storage space and may be ableto be rendered faster and/or more efficiently.

Embodiments of the present disclosure are explained with reference tothe accompanying drawings.

FIG. 1 is a diagram representing an example environment 100 related togenerating a simulated driving environment, arranged in accordance withat least one embodiment described in the present disclosure. Theenvironment 100 may include a filtering module 110 configured to filtermap data 102 based on how autonomous-vehicle software may use inputsthat may be obtained based on the map data 102. In some embodiments, thefiltering module 110 may filter the map data 102 based onautonomous-vehicle data 104. The filtering module 110 may outputfiltered map data 112. In these or other embodiments, the environment100 may include an environmental representation-creation module 120(creation module 120) configured to generate a 3-D environmentalrepresentation 122 (representation 122) based on the filtered map data112.

In some embodiments, the map data 102 may include data describing ageographic area. The map data 102 may describe a geographic area in thereal world. The map data 102 may include multiple layers of datadescribing the same geographic area. For example, the map data 102 mayinclude road data 103 describing roads, general area data 105 describingthe appearance and size of buildings and other areas between the roads,and terrain data 107 describing changes in elevation. The road data 103,the general area data 105, and the terrain data 107 may all describe thesame geographic area and may together provide a more completedescription of the geographic area than any one of them may providealone.

The map data 102 may include information found on a street map orinformation observable from an aerial photo or a satellite photo.Additionally or alternatively, the map data 102 may include informationobtained by a vehicle traveling on the roads of the geographic area andcapturing images of the geographic area. The map data 102 may includedetailed, for example, high definition (HD) information. For instance,the map data 102 may include LIDAR data in a format such as, forexample, a point cloud. In some embodiments, each data point in thepoint cloud may include positional information (e.g., x-, y-, andz-coordinates on three-dimensional axes) as well as intensityinformation (e.g., reflective information). The points in the pointcloud may correspond to elements in the geographic setting associatedwith the HD map data. In some embodiments, the intensity information mayfacilitate the differentiation of the points in the point cloud intodifferent data topics that may correspond to different types ofelements. For example, based on the intensity information, points in thepoint cloud may be characterized into data topics such as “lane lines,”“trees,” “curbs,” “road edges,” “sidewalks” “roads,” “trees,” or othercategories. In some embodiments, points in the point cloud may beorganized into data topics based on the intensity information, thepositional information, and/or other information. In some embodiments, aLIDAR point cloud may include or may be supplemented by GlobalPositioning System (GPS) data. For example, a center of the point cloudmay be associated with particular GPS coordinates and the positionalinformation (i.e., the x-, y-, and z-coordinates) may be referenced fromthe center.

In some embodiments, the map data 102 may be or may include photographsof an intersection including resolution on a centimeter scale or an evenmore granular scale. In these and other embodiments, photographs may beused to enhance the LIDAR point cloud. The map data 102 may includeother highly accurate HD data, for example, the location of theintersection in the map data 102 may be an accurate representation ofwhere the intersection is in the real world to within a centimeter.Additionally or alternatively, the map data 102 may includeless-accurate information about other aspects of the geographic area.For instance the map data 102 may not distinguish between individualtrees in a copse of trees.

The map data 102 may include the road data 103 of roads in thegeographic area. The road data 103 may include locations of elementsincluding: roads, sidewalks, lane lines, curbs, sidewalk edges,intersections, road signs, traffic lights, cross walks, parking lots,parking structures, bridges, overpasses, tunnels, railroad crossings,islands between lanes, bike lanes, traffic signs, and school zones.

Additionally or alternatively, the map data 102 may include the generalarea data 105 that may include information regarding areas between theroads of the geographic area. The general area data 105 may includeinformation about buildings, parks, fields, and outdoor areas. Thegeneral area data 105 may include images of buildings or images takenfrom the road of areas between the roads. The general area data 105 mayinclude 3-D representations of objects, for example, buildings, firehydrants, and light posts.

Additionally or alternatively, the map data 102 may include the terraindata 107 of the geographic area. The terrain data 107 may includeelevation data of the geographic area, for example, a topographical mapof the geographic area.

In some embodiments, the autonomous-vehicle data 104 may includeinformation about inputs of autonomous-vehicle software and how theinputs may be used by the autonomous-vehicle software. For example, theautonomous-vehicle data 104 may include information about sensors on anautonomous vehicle. For instance, information about what kinds ofsensors are used, resolution of the sensors, depth of focus of thesensors, pointing direction of the sensors, frame-rate of the sensors,and/or scan-rate of the sensors and/or emitters. Additionally oralternatively, the autonomous-vehicle data 104 may include informationabout how data from the sensors is used in the autonomous-vehiclesoftware. For example, the autonomous-vehicle software may be configuredto use high-resolution image processing techniques on high-resolutionimages of the road in front of the autonomous vehicle at a highfrequency of recurrence. As another example, the autonomous-vehiclesoftware may be configured to use low-resolution image processingtechniques on portions of images that include space that is more thanfive meters from the road on which the autonomous vehicle is travelling.

The filtering module 110 may include code and routines configured toenable a computing device to perform one or more operations with respectto filtering map data 102 to obtain the filtered map data 112.Additionally or alternatively, the filtering module 110 may beimplemented using hardware including a processor, a microprocessor(e.g., to perform or control performance of one or more operations), afield-programmable gate array (FPGA), or an application-specificintegrated circuit (ASIC). In some other instances, the filtering module110 may be implemented using a combination of hardware and software. Inthe present disclosure, operations described as being performed by thefiltering module 110 may include operations that the filtering module110 may direct a corresponding system to perform.

The filtered map data 112 may include map data selected from the mapdata 102. As such, the filtered map data 112 may include the map data102 and/or a selected (e.g., filtered) a subset of the map data 102. Thefiltered map data 112 may include filtered road data 113, which mayinclude the road data 103 and/or a selected (e.g., filtered) subset ofthe road data 103. Additionally or alternatively, the filtered map data112 may include filtered general area data 115, which may include thegeneral area data 105 or a selected (e.g., filtered) subset of thegeneral area data 105. Additionally or alternatively, the filtered mapdata 112 may include filtered terrain data 117, which may include theanalysis module 107 or a selected (e.g., filtered) subset of theanalysis module 107.

In some embodiments, some of the information of the filtered map data112 may have been filtered more by the filtering module 110 than otherinformation such that different types of information in the filtered mapdata 112 may have different levels of accuracy and/or resolution. Forexample, the filtered road data 113 may retain a high amount of itsinitial accuracy and/or resolution while filtered general area data 115may retain a lower amount of its initial accuracy and/or resolution.

The filtering module 110 may be configured to select map data 102 forinclusion in the filtered map data 112 based on a relationship betweenthe map data 102 and the autonomous-vehicle data 104. More specifically,the filtering module 110 may be configured to select (which may includefiltering) road data 103 of the map data 102, according to a firstdegree of accuracy based on a relationship between the road data 103 andhow the autonomous-vehicle software may use information derived from theroad data 103. (Information derived from the road data 103 may includesimulated images or other sensor readings based on a 3-D environmentalrepresentation of roads or other elements of the road data 103.)

As an example of selection of the road data 103 by the filtering module110, a relationship may describe that the autonomous-vehicle software isconfigured to process images with image processing techniques providinga first level of accuracy. The road data 103 may include images thatinclude a higher level of accuracy than the first level of accuracy. Inthese cases, the road data 103 may be filtered such that the informationretained corresponds to the first level or accuracy.

In some embodiments, the filtering may depend on the environment fromwhich the HD map data is obtained. For example, in some embodiments, theHD map data may be from a highway. In these and other embodiments,objects in the environment may be of relatively little importance and/ormay be relatively static. For example, in some locations, a highway maybe relatively straight with few turns. In these and other embodiments,more of the data may be filtered without losing any accuracy. Forexample, if the HD map data includes a point cloud with 100,000 points,50,000 of the points may be filtered when the HD map data correspondswith a highway.

Alternatively, in some embodiments, the HD map data may be from an urbanenvironment. In these and other embodiments, objects in the environmentmay be of relatively great importance and/or may not be relativelystatic. For example, in some locations, urban streets may include manyturns, objects, or other details. In these and other embodiments, fewpoints may be filtered. For example, if the HD map data includes a pointcloud with 100,000 points, 10,000 of the points may be filtered when theHD map data corresponds with an urban area.

In some embodiments, different data topics may include different amountsof filtering. For example, in some embodiments, points in a “road edge”data topic may be filtered to a greater degree than points in a “laneline” data topic. Alternatively or additionally, in some embodiments,points in a “dashed line” data topic may be filtered to a greater degreethan points in a “solid line” data topic. In some embodiments, differentdata topics may be filtered in different degrees depending on theenvironment. For example, in a highway environment, points in a “roadedge” data topic may be filtered less than other points. Alternatively,in an urban environment, points in a “curbs” data topic may be filteredless than other points.

As an example of selection of the road data 103 by the filtering module110, a relationship may describe that the autonomous-vehicle softwaremay determine and/or control the position of tires of the autonomousvehicle relative to line lanes and/or curbs to within one centimeter. Inthese cases, the filtering module 110 may be configured to select roaddata 103 including all road data having one centimeter accuracy, orone-half centimeter accuracy. In some embodiments, this may mean theselection of all available road data 103 for inclusion in the filteredroad data 113 without filtering or removing any of the accuracy and/orresolution of the road data.

In some embodiments, the autonomous-vehicle software may be configuredto determine and/or control the position of the vehicle (e.g., the tiresof the vehicle) with a high degree of accuracy relative to elements ofthe road data. For example, the autonomous-vehicle software may spendmore processing time determining where the vehicle is relative to thelane lines than the processing time spent determining where the vehicleis relative to a building beside the road. As such, theautonomous-vehicle software may determine where the autonomous vehicleis relative to the lane lines with a higher degree of accuracy than thedegree of accuracy used in determining the where the autonomous vehicleis relative to the building.

In these or other embodiments, the filtering module 110 may beconfigured to select general area data 105 of the map data 102,according to a second degree of accuracy (which may be lower than thefirst degree of accuracy) based on a relationship between the generalarea data 105 and how the autonomous-vehicle software may useinformation derived from the general area data 105. (Information derivedfrom the general area data may include simulated images or other sensorreadings based on a 3-D environmental representation of objects of thegeneral area data 105.)

As an example of selection of general area data 105 by the filteringmodule 110, one of the relationships may describe that theautonomous-vehicle software ignores inputs and/or the autonomous-vehiclesensors are not angled to receive inputs that are more than six metersabove the ground. Based on this relationship, the filtering module 110may exclude (or include at a limited resolution and/or accuracy) generalarea data 105 regarding anything higher than six meters from the groundin the filtered general area data 115. As an example of selection of thegeneral area data 105 by the filtering module 110, a relationship maydescribe that the autonomous-vehicle software is configured to processall images at all angles with image processing techniques providing acertain level of accuracy. The general area data 105 may include imagesthat include a lower level of accuracy than the certain level ofaccuracy. In these cases, the general area data 105 may be selectedwithout any filtration for inclusion in the filtered general area data115.

In these or other embodiments, the filtering module 110 may beconfigured to select general area data 105 of the map data 102,according to a second degree of accuracy (which may be lower than thefirst degree of accuracy) based on a relationship between the generalarea data 105 and the road data 103. For example, one of therelationships may describe that the autonomous-vehicle sensors can onlyfocus on objects within 2 meters of the sensors. Based on an assumptionthat the autonomous vehicle will travel on roads, it may be determinedthat objects more than three meters from any road do not require highresolution and/or accuracy. Based on this determination, the filteringmodule 110 may reduce the resolution and/or accuracy in filtered generalarea data 115 of objects that are more than three meters from any road.

In these or other embodiments, the filtering module 110 may beconfigured to select terrain data 107 of the map data 102, according toa third degree of accuracy (which may be the same as or lower than thefirst degree of accuracy) based on a relationship between the terraindata 107 and how the autonomous-vehicle software may use informationderived from the terrain data 107. Information derived from the terraindata 107 may include simulated images or other sensor readings (e.g.,readings from an accelerometer, a gyroscope, and/or shocks of theautonomous vehicle) based on a 3-D environmental representation ofterrain of the terrain data 107. For example, a simulated 3-Drepresentation of a road based on terrain data 107 may include inclinesand declines based on inclines and declines in the terrain data 107.Information presented to the autonomous-vehicle software may reflect theinclines and declines. For example, images of the road and/or generaarea may be adjusted to reflect the inclines and declines. For example,an image of a road may be stretched or shrunken such that it fits a 3-Drepresentation of the road. Additionally or alternatively, gyroscopicand/or accelerometer data may reflect the inclines and declines. Forexample, the simulated gyroscopic data may indicate that the car isoriented pointing upward when the simulated autonomous vehicle is on asimulated incline.

As an example of selection of terrain data 107 by the filtering module110, in some areas, the elevation data of the analysis module 107 mayhave a higher degree of accuracy than the road data 103. In such cases,the filtering module 110 may reduce the accuracy of elevation data 107to be no more than the accuracy of the map data 103.

In some embodiments, the filtering module 110 may take theautonomous-vehicle data 104 as an input. In these or other embodiments,the filtering module 110 may be configured to determine relationshipsbetween the map data 102 and how information derived from the map data102 may be used by the autonomous-vehicle software based on theautonomous-vehicle data 104. Additionally or alternatively, thefiltering module 110 may obtain the relationships between the map data102 and how information derived from the map data 102 may be used by theautonomous-vehicle software from another source. For example, a user maydefine the relationships and provide the relationships to the filteringmodule 110. In these or other embodiments, the filtering module 110 maynot require the autonomous-vehicle data 104 as an input. Additionally oralternatively, the filtering module 110 may obtain filters that arebased on the relationships between the map data 102 and how informationderived from the map data 102 may be used by the autonomous-vehiclesoftware from another source. For example, a user, or another system maygenerate filters based on the relationships. The filtering module 110may use the filters generated by the user or the other system.

The environmental representation-creation module 120 may include codeand routines configured to enable a computing device to perform one ormore operations with respect to generating the 3-D environmentalrepresentation 122 based on the filtered map data 112. Additionally oralternatively, the environmental representation-creation module 120 maybe implemented using hardware including a processor, a microprocessor(e.g., to perform or control performance of one or more operations), afield-programmable gate array (FPGA), or an application-specificintegrated circuit (ASIC). In some other instances, the environmentalrepresentation-creation module 120 may be implemented using acombination of hardware and software. In the present disclosure,operations described as being performed by the environmentalrepresentation-creation module 120 may include operations that theenvironmental representation-creation module 120 may direct acorresponding system to perform.

In some embodiments, the environmental representation-creation module120 may be configured to generate the 3-D environmental representation122. The environmental representation-creation module 120 may generatethe 3-D environmental representation 122 according to any suitable 3-Dmodeling technique. In some embodiments, the environmentalrepresentation-creation module 120 may use the filtered map data 112 asinput data in the generation of the 3-D environmental representation122. For example, the environmental representation-creation module 120may base the 3-D environmental representation 122 on the filtered mapdata 112. For example, the 3-D environment of the 3-D environmentalrepresentation 122 may represent the geographic area represented by thefiltered map data 112.

In some embodiments, the 3-D environmental representation 122 mayinclude a 3-D model of one or more objects in the geographic area asdescribed by the filtered map data 112. For example, the 3-Denvironmental representation 122 may include a complete 3-D model of thesimulated driving environment.

Modifications, additions, or omissions may be made to the environment100 of FIG. 1 without departing from the scope of the presentdisclosure. For example, the environment 100 may include more or fewerelements than those illustrated and described in the present disclosure.For instance, in some embodiments, the autonomous-vehicle data 104 maybe omitted. In these embodiments, relationships derived from theautonomous-vehicle data 104 may be included in the filtering module 110,or may be provided to the filtering module 110 in some other format.

FIG. 2 is a diagram representing another example environment 200 relatedto generating a simulated driving environment, arranged in accordancewith at least one embodiment described in the present disclosure. Theenvironment 200 may include a filtering module 222 that includes a firstfiltering module 220A, a second filtering module 220B, and a thirdfiltering module 220C and a mesh generation module 250.

The filtering module 222 may be configured to perform one or morefiltering operations analogous to those described above with respect tothe filtering module 110 of FIG. 1 . Additionally or alternatively, thefiltering module 222 may be configured to perform filtering operationswith respect to HD mapping data 212, which may include first HD mappingdata 210A, second HD mapping data 210B, and third HD mapping data 210C.For instance, each of the filtering modules 220 may be configured toreceive corresponding HD mapping data. For example, the first filteringmodule 220A may be configured to receive the first HD mapping data 210A,the second filtering module 220B may be configured to receive the secondHD mapping data 210B, and the third filtering module 220C may beconfigured to receive the third HD mapping data 210C.

In some embodiments, the HD mapping data 212 may include data describinga geographic area. The HD mapping data 212 may describe a geographicarea in the real world. The HD mapping data 212 may include data inmultiple different data topics. For example, the HD mapping data 212 mayinclude road data, general area data, and terrain data such as thatdescribed above with respect to FIG. 1 .

In some embodiments, the HD mapping data 212 may include a point cloudgenerated from data from LIDAR sensors. In these and other embodiments,the point cloud may include thousands of points, millions of points,billions of points, or more points. In some embodiments, as discussedabove, each data point in the point cloud may include positionalinformation (e.g., x-, y-, and z-coordinates on three-dimensional axes)as well as intensity information (e.g., reflective information). As alsodiscussed above, the points in the point cloud may correspond toelements in the geographic setting associated with the HD mapping data212. In some embodiments, the intensity information may facilitate thedifferentiation of the points in the point cloud into different datatopics.

In some embodiments, data topics may be nested. For example, lane linesmay be a data topic. Dashed lane lines, solid lane lines, and doublesolid lane lines may be subtopics of the lane lines data topic.Alternatively or additionally, yellow lane lines and white lane linesmay be subtopics of the lane lines data topic and/or the dashed lanelines, solid lane lines, and double solid lane lines subtopics.

As mentioned above, in some embodiments, points in the point cloud maybe organized into data topics based on the intensity information, thepositional information, and/or other information. In some embodiments,the point cloud may include or may be supplemented by Global PositioningSystem (GPS) data. For example, a center of the point cloud may beassociated with particular GPS coordinates and the positionalinformation (i.e., the x-, y-, and z-coordinates) may be referenced fromthe center with the particular GPS coordinates.

In some embodiments, the HD mapping data 212 may include highly accurateand/or detailed information regarding particular geographic and/ortraffic elements but may not include highly accurate and/or detailedinformation regarding other elements. For example, the HD mapping data212 may include highly accurate data, for example, concerning thelocation of an intersection on a road and the location of lane lines ona road but may not distinguish between individual trees in a copse oftrees.

As indicated above, in some embodiments, the HD mapping data 212 mayinclude road data, general area data, and/or terrain data.

The filtering module 222 (including the filtering modules 220) mayinclude code and routines configured to enable a computing device toperform one or more operations with respect to filtering HD mapping data212 to obtain filtered HD mapping data. Additionally or alternatively,the filtering module 222 may be implemented using hardware including aprocessor, a microprocessor (e.g., to perform or control performance ofone or more operations), a field-programmable gate array (FPGA), or anapplication-specific integrated circuit (ASIC). In some other instances,the filtering module 222 may be implemented using a combination ofhardware and software. In the present disclosure, operations describedas being performed by the filtering module 222 may include operationsthat the filtering module 222 may direct a corresponding system toperform.

As indicated above, in some embodiments, the filtering module 222 may beconfigured to filter the HD mapping data 212 according to one or morefiltering operations such as described above with respect to thefiltering module 110 of FIG. 1 . In these or other embodiments, each ofthe filtering modules 220 may be associated with a particular datatopic. For example, in some embodiments, the first filtering module 220Amay be associated with lane lines. In these and other embodiments, thefirst filtering module 220A may be configured to receive HD mapping data212 for a lane lines data topic, e.g., the first HD mapping data 210A,and to filter some of the HD mapping data 212 to generate filtered HDmapping data for lane lines.

Similarly, the second filtering module 220B may be associated withcurbs. In these and other embodiments, the second filtering module 220Bmay be configured to receive HD mapping data 212 for a curbs data topic,e.g., the second HD mapping data 210B, and to filter some of the HDmapping data 212 to generate filtered HD mapping data for curbs.

Similarly, the third filtering module 220C may be associated with roadedges. In these and other embodiments, the third filtering module 220Cmay be configured to receive HD mapping data 212 for a road edge datatopic, e.g., the third HD mapping data 210C, and to filter some of theHD mapping data 212 to generate filtered HD mapping data for road edges.

Lane lines, curbs, and road edges are merely examples of data topicsthat may be described in HD mapping data 212 and other topics areconsidered. For example, in some embodiments, the HD mapping data 212may include any number of different types of HD mapping data 210 thatcorresponds to any number of different data topics. Additionally oralternatively, the filtering module 222 may include more filteringmodules 220 such as, for example, ten filtering modules 220, twentyfiltering modules 220, one hundred filtering modules 220, etc.Alternatively, in some embodiments, the filtering module 222 may includefewer filtering modules 220. Further although described as eachcorresponding to different data topics, in some embodiments, one or moreof the filtering modules 220 may each correspond to any number of datatopics. Further, in some embodiments, the environment 200 may notinclude the filtering module 222 or any of its sub-filtering modules220.

In some embodiments, the autonomous-vehicle software may be configuredto determine and/or control the position of the vehicle (e.g., the tiresof the vehicle) with a high degree of accuracy relative to elements ofthe road data. For example, the autonomous-vehicle software may spendmore processing time determining where the vehicle is relative to thelane lines than the processing time spent determining where the vehicleis relative to a building beside the road. As such, theautonomous-vehicle software may determine where the autonomous vehicleis relative to the lane lines with a higher degree of accuracy than thedegree of accuracy used in determining the where the autonomous vehicleis relative to the building.

In some embodiments, the filtering module 222 may filter the HD mappingdata 212 depending on the environment (e.g., rural, highway, urban,etc.) from which the HD mapping data 212 is obtained such as describedabove. In these and other embodiments, the first filtering module 220A,the second filtering module 220B, and the third filtering module 220Cmay employ similar filtering for the first HD mapping data 210A, thesecond HD mapping data 210B, and the third HD mapping data 210C,respectively, based on the environment to which the HD mapping data 210corresponds.

As discussed above, in some embodiments, different data topics mayinclude different amounts of filtering. For example, in these and otherembodiments, the first filtering module 220A may filter the first HDmapping data 210A to a greater degree than the second filtering module220B may filter the second HD mapping data 210B. For example, asdiscussed above, in some embodiments, points in a “road edge” data topicmay be filtered to a greater degree than points in a “lane line” datatopic. Alternatively or additionally, in some embodiments, points in a“dashed line” data topic may be filtered to a greater degree than pointsin a “solid line” data topic.

In some embodiments, filtering the HD mapping data 212 may depend onboth the environment and on the data topic. For example, in a highwayenvironment, a road edge data topic may be of relatively greaterimportance than other data topics. Thus, when the environment is ahighway, a filtering module 220 associated with road edges may filterthe data less than other filtering modules 220. Alternatively, when theenvironment is an urban area, a curbs data topic may be of relativelygreater interest than other data topics. Thus, when the environment isan urban area, a filtering module 220 associated with curbs may filterthe data less than other filtering modules 220.

The filtering modules 220 may provide filtered HD mapping data of eachthe HD mapping data 210 to the mesh generation module 250. In someembodiments, the HD mapping data 212 may be separated into distinctfiles. In these and other embodiments, the filtered HD mapping data fromthe filtering module 222 may similarly be separated into distinct filesfor each data topic. In these and other embodiments, separating inputfiles based on data topics may facilitate the creation of a mesh by themesh generation module 250. For example, separation of distinct datatopics into different files may make the mesh generation module 250 moreefficient.

In some embodiments, the mesh generation module 250 may be included inor an example of the environmental creation module 120 of FIG. 1 . Themesh generation module 250 may include code and routines configured toenable a computing device to perform one or more operations with respectto generating one or more meshes from the HD mapping data 212 and/orfrom filtered HD mapping data from the filtering module 222.Additionally or alternatively, the mesh generation module 250 may beimplemented using hardware including a processor, a microprocessor(e.g., to perform or control performance of one or more operations), afield-programmable gate array (FPGA), or an application-specificintegrated circuit (ASIC). In some other instances, the mesh generationmodule 250 may be implemented using a combination of hardware andsoftware. In the present disclosure, operations described as beingperformed by the mesh generation module 250 may include operations thatthe mesh generation module 250 may direct a corresponding system toperform.

Although depicted as a single mesh generation module 250, in someembodiments, there may be more than one mesh generation module 250. Forexample, in these and other embodiments, there may be a mesh generationmodule 250 for each data topic in the HD mapping data 212. That is, insome embodiments, there may be an equal number of HD mapping data 210,filtering modules 220, and mesh generation modules 250.

In some embodiments, the mesh generation module 250 may generate a meshfor each distinct instance of each data topic of the point cloud of theHD mapping data 212. In these and other embodiments, a mesh may includea 3-D representation of the points in the point cloud with polygonsconnecting the data points. For example, in some embodiments, the pointsin the point cloud may be distinct points and may not be connected toany other points. In these and other embodiments, a mesh may include thepoints of the point cloud as vertices of one or more polygonsrepresenting sides of objects in a 3-D space. For example, an object,such as a lane line, may be represented by thousands of discrete pointsin a point cloud. A mesh generated from the points of the point cloudrepresenting the lane line might include lines connecting the pointsinto the shape of the lane line. In these and other embodiments, thepoints may be connected by polygons, which may give the lane linessurfaces and/or a 3-D shape including thickness.

In some embodiments, after generation of a mesh, a texture may beapplied to the mesh. In these and other embodiments, the texture mayinclude colors, patterns, surface details, and/or other information totransform the mesh into a more realistic 3-D representation of anobject. In these and other embodiments, different data topics maycorrespond with different textures. For example, in some embodiments,yellow lane lines and white lane lines may represent different datatopics and may be associated with different textures. In these and otherembodiments, the different textures may correspond with differentreal-world examples of the data topics. For example, for yellow lanelines, different textures may include “clean lines,” in which the yellowlane lines are clean and distinct; “faded lines,” in which the yellowlane lines appear faded; and “cracked lines,” in which part of theyellow lane lines may have lost their color.

Application of multiple textures to a mesh is also contemplated. In someembodiments, multiple textures may be applied to a single mesh. Forexample, in some embodiments, textures may be applied to specificpolygons (such as, for example, specific triangles) within a single meshbut may not be applied to other polygons in the mesh. For example, afirst texture may be applied to some polygons in a particular mesh and asecond texture may be applied to other polygons in the particular mesh.Alternatively or additionally, in some embodiments, multiple texturesmay be layered on particular mesh polygons. For example, a first texturemay be applied to all polygons in a particular mesh and a second texturemay be layered on top of the first texture for some of the polygons inthe particular mesh. Alternatively or additionally, in some embodiments,textures may not be applied to some polygons in a particular mesh whileone or more textures may be applied to other polygons in the particularmesh.

In some embodiments, providing different textures for differentinstances of a single data topic may result in a more robust simulationenvironment. For example, when an autonomous vehicle is driving along aroad, not all instances of a particular data topic will appear the same.As discussed above, lane lines may be in different states of disrepair,with some lane lines faded, some well maintained, and others missing atpoints. By providing a method to easily alter the appearance of someinstances of a data topic while maintaining the appearance of otherinstances of the same data topic, the simulation environment may providea better test of whether an autonomous vehicle will correctly recognizelane lines or other objects even when the objects differ in appearancefrom the standard version of the object or when the objects may differover the course of a relatively short distance (e.g., where lane linesare currently being painted and a first segment of the lane lines arefreshly painted and a second segment of the lane lines are faded and/orcracked.

In addition to applying different textures to different instances of aparticular data topic, the mesh generation module 250 may applydifferent textures to instances of different data topics. In these andother embodiments, the textures that may be applied to one data topicmay differ from the textures that may be applied to another data topic.For example, textures associated with road edge data topics may bedifferent from those associated with curb data topics.

In addition to generating a mesh from HD mapping data 212, the meshgeneration module 250 may be configured to generate meshes from adatabase 230 and/or from scenarios 240. In these and other embodiments,the database 230 may include information about buildings and/or otherobjects that may not be included in HD mapping data 212 from LIDARsensors and/or about which the HD mapping data 212 may includeincomplete information. For example, traditional LIDAR sensors onautomobiles may be directed horizontally or below horizontal. Thus,traditional LIDAR data in the form of a point cloud may includeincomplete information about objects, such as buildings, over the heightof an automobile.

The scenario data 240 may include information about trees, shrubbery,and/or other objects that may similarly be incompletely and/orinaccurately captured in LIDAR data. Alternatively or additionally, insome embodiments, the scenario data 240 may be associated with aparticular scenario in which it is desired to test an autonomousvehicle.

The database 230 and the scenario data 240 may thus supplement theinformation from the HD mapping data 212 to help the mesh generationmodule 250 generate a more realistic and/or robust building for asimulation environment. In some embodiments, the database 230 and/or thescenario data 240 may not be data in the form of a point cloud, unlikethe HD mapping data 212. In these and other embodiments, the meshgeneration module 250 may not generate a mesh from the database 230 orthe scenario data 240. In these and other embodiments, the database 230and/or the scenario data 240 may include data in the form of a layerwhich may be positioned on top of one or more meshes generated by themesh generation module 250. For example, in some embodiments, each meshgenerated by the mesh generation module 250 may be a layer in asimulation environment 260. The database 230 may also include severallayers for the simulation environment 260. For example, in someembodiments, each building in the database 230 and/or each static objectsuch as, for example, a tree, may be a separate layer in the simulationenvironment 260.

In some embodiments, the mesh generation module 250 may further add oneor more decals to the textures of the meshes. For example, in someembodiments, the decals may include details associated with a data topicand/or a texture applied to a particular instance of a data topic. Forexample, in some embodiments, details indicating a crack in a road maybe added to a texture for a road surface as a decal. Alternatively oradditionally, in some embodiments, decals indicating a pothole in a roadmay be added. Alternatively or additionally, in some embodiments, adecal may depict other road qualities that may be tailored to a specificsimulation environment in order to test the robustness of an autonomousvehicle against particular targeted deficiencies.

The simulation environment 260 may include a combination of the meshesgenerated for the different data topics to represent the correspondingenvironment and its elements such as a variety of lane lines, curbs,road edges, buildings, trees, and/or other information associated withautonomous vehicles and traffic. In some embodiments, the simulationenvironment 260 may include multiple layers, including a base layer anddifferent layers for each mesh generated by the mesh generation module250 and layers for the database 230 and the scenario data 240. Thesimulation environment 260 may include a 3-D model of one or moreobjects in the geographic area as described by the HD mapping data 212.In some embodiments, the simulation environment 260 may be an example ofthe environmental representation 122 of FIG. 1

Modifications, additions, or omissions may be made to the environment200 of FIG. 2 without departing from the scope of the presentdisclosure. For example, the environment 200 may include more or fewerelements than those illustrated and described in the present disclosure.For instance, the database 230 and/or the scenario data 240 may beomitted. In these embodiments, information related to buildings and/orstatic objects may be included in the HD mapping data 212 or may beomitted.

FIG. 3 illustrates an example point cloud of HD mapping data, accordingto at least one embodiment of the present disclosure. As depicted inFIG. 3 , the HD mapping data may include a first sidewalk 310A and asecond sidewalk 310B (collectively the sidewalks 310), a first dashedwhite line 320A and a second dashed white line 320B (collectively thedashed white lines 320), and solid yellow double center line 330. Inthese and other embodiments, the sidewalks 310, the dashed white lines320, and the solid yellow double center line 330 may each represent adistinct data topic. In these and other embodiments, the first sidewalk310A and the second sidewalk 310B may each represent a distinct instanceof the sidewalks 310 data topic. Similarly, the first dashed white line320A and the second dashed white line 320B may each represent a distinctinstance of the dashed white lines 320 data topic. The solid yellowdouble center line 330 may also represent a distinct instance of thesolid yellow double center line 330 data topic.

In some embodiments, each object in the point cloud may include multiplepoints. For example, in some embodiments, each of the first sidewalk310A, the second sidewalk 310B, the first dashed white line 320A, thesecond dashed white line 320B, and the solid yellow double center line330 may include thousands of points, millions of points, or any numberof points.

In some embodiments, the point cloud may also include additional pointsthan those illustrated in FIG. 3 . For example, in some embodiments,points may also be located between the first sidewalk 310A and the firstdashed white line 320A (i.e., points associated with the road), orbetween the first dashed white line 320A and the solid yellow doublecenter line 330 (i.e., points associated with the road). In these andother embodiments, each of the points associated with an object mayinclude positional information (represented in FIG. 3 as the location ofthe point) and intensity information (represented in FIG. 3 as thedistinction between objects). For example, points in the point cloud maybe determined to be associated with the first sidewalk 310A and not aroad based on the intensity associated with the point. Additionally oralternatively, in some embodiments, points may be determined to beassociated with a particular data topic or a particular instance of adata topic based on the proximity of the points to other points of asimilar data topic. For example, the points of the first dashed whiteline 320A and the second dashed white line 320B may both be determinedto be associated with the dashed white line 320 data topic based on theintensity of the points but may be determined to be distinct instancesof the dashed white line 320 data topic based on the distance betweenthe points and an approximate direction of the group of points.

FIG. 4 illustrates example meshes, according to at least one embodimentof the present disclosure. In some embodiments, the meshes of FIG. 4 maybe generated from a point cloud such as the point cloud depicted in FIG.4 . As depicted in FIG. 4 , the meshes may include a first sidewalk mesh410A and a second sidewalk mesh 410B (collectively the sidewalks 410), afirst dashed white line mesh 420A and a second dashed white line mesh420B (collectively the dashed white lines 420), and solid yellow doublecenter line mesh 430. In these and other embodiments, the sidewalks 410,the dashed white lines 420, and the solid yellow double center line 430may each represent a distinct data topic. In these and otherembodiments, the first sidewalk mesh 410A and the second sidewalk mesh410B may each represent a distinct instance of the sidewalks 410 datatopic. Similarly, the first dashed white line mesh 420A and the seconddashed white line mesh 420B may each represent a distinct instance ofthe dashed white lines 420 data topic. The solid yellow double centerline mesh 430 may also represent a distinct instance of the solid yellowdouble center line 430 data topic.

In some embodiments, each mesh may include multiple polygons, eachpolygon formed from points in the point cloud. For example, in someembodiments, as compared to the corresponding objects in FIG. 3 , eachobject in FIG. 4 may include surfaces representing sides of an object in3-D space. For example, the first sidewalk mesh 410A may include sideand top surfaces, the second sidewalk mesh 410B may include side and topsurfaces, the first dashed white line mesh 420A may include side and topsurfaces, the second dashed white line mesh 420B may include side andtop surfaces, and the solid yellow double center line 430 may includeside and top surfaces. In these and other embodiments, each mesh mayinclude many polygons, some of which may not be contiguous with otherpolygons. For example, while the first sidewalk mesh 410A is depictedwith a single rectangular polygon as a top surface, it may also includemany distinct polygons as a top surface. The first dashed white linemesh 420A may include multiple noncontiguous polygons. For example, asdepicted, the first dashed white line mesh 420A may include three setsof noncontiguous polygons.

In some embodiments, additional meshes may be generated from a pointcloud. For example, in some embodiments, meshes may also be generatedbetween the first sidewalk mesh 410A and the first dashed white linemesh 420A, for example as a road mesh. For example, in some embodiments,a point cloud may include points between the first sidewalk mesh 410Aand the first dashed white line mesh 420A corresponding to, for example,a road.

In some embodiments, the meshes of FIG. 4 may include additionalelements. For example, as discussed above, the meshes of distinctinstances of data topics may include textures. As an example, the firstdashed white line mesh 420A may include a “faded” texture while thesecond dashed white line mesh 320B may include “freshly painted”texture.

FIG. 5 illustrates a block diagram of an example computing system 500,according to at least one embodiment of the present disclosure. Thecomputing system 500 may be configured to implement or direct one ormore operations associated with any one of the modules discussed in thepresent disclosure. The computing system 500 may include a processor502, a memory 504, and a data storage 506. The processor 502, the memory504, and the data storage 506 may be communicatively coupled.

In general, the processor 502 may include any suitable special-purposeor general-purpose computer, computing entity, or processing deviceincluding various computer hardware or software modules and may beconfigured to execute instructions stored on any applicablecomputer-readable storage media. For example, the processor 502 mayinclude a microprocessor, a microcontroller, a digital signal processor(DSP), an application-specific integrated circuit (ASIC), aField-Programmable Gate Array (FPGA), or any other digital or analogcircuitry configured to interpret and/or to execute program instructionsand/or to process data. Although illustrated as a single processor inFIG. 5 , the processor 502 may include any number of processorsconfigured to, individually or collectively, perform or directperformance of any number of operations described in the presentdisclosure. Additionally, one or more of the processors may be presenton one or more different electronic devices, such as different servers.

In some embodiments, the processor 502 may be configured to interpretand/or execute program instructions and/or process data stored in thememory 504, the data storage 506, or the memory —504 and the datastorage 506. In some embodiments, the processor 502 may fetch programinstructions from the data storage 506 and load the program instructionsin the memory 504. After the program instructions are loaded into memory504, the processor 502 may execute the program instructions.

For example, in some embodiments, one or more of the above mentionedmodules may be included in the data storage 506 as program instructions.The processor 502 may fetch the program instructions of a correspondingmodule from the data storage 506 and may load the program instructionsof the corresponding module in the memory 504. After the programinstructions of the corresponding module are loaded into memory 504, theprocessor 502 may execute the program instructions such that thecomputing system may implement the operations associated with thecorresponding module as directed by the instructions.

The memory 504 and the data storage 506 may include computer-readablestorage media for carrying or having computer-executable instructions ordata structures stored thereon. Such computer-readable storage media mayinclude any available media that may be accessed by a general-purpose orspecial-purpose computer, such as the processor 502. By way of example,and not limitation, such computer-readable storage media may includetangible or non-transitory computer-readable storage media includingRandom Access Memory (RAM), Read-Only Memory (ROM), ElectricallyErasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-OnlyMemory (CD-ROM) or other optical disk storage, magnetic disk storage orother magnetic storage devices, flash memory devices (e.g., solid statememory devices), or any other storage medium which may be used to carryor store particular program code in the form of computer-executableinstructions or data structures and which may be accessed by ageneral-purpose or special-purpose computer. Combinations of the abovemay also be included within the scope of computer-readable storagemedia. Computer-executable instructions may include, for example,instructions and data configured to cause the processor 502 to perform acertain operation or group of operations.

Modifications, additions, or omissions may be made to the computingsystem 500 without departing from the scope of the present disclosure.For example, in some embodiments, the computing system 500 may includeany number of other components that may not be explicitly illustrated ordescribed.

FIG. 6 is a flowchart of an example method 600 of generating a simulateddriving environment, according to at least one embodiment described inthe present disclosure. The method 600 may be performed by any suitablesystem, apparatus, or device with respect to the simulated drivingenvironment. For example, one or more of the filtering module 110 ofFIG. 1 , the filtering module 222 of FIG. 2 , the environmentalrepresentation-creation module 120 of FIG. 1 , the mesh generationmodule 250 of FIG. 2 , or the computing system 500 of FIG. 5 (e.g., asdirected by one or more modules) may perform one or more of theoperations associated with the method 600. Although illustrated withdiscrete blocks, the steps and operations associated with one or more ofthe blocks of the method 600 may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the particularimplementation.

The method 600 may begin at block 610, where map data of a geographicarea may be obtained. The map data may include road data of roadsincluded in the geographic area and general area data that relates tosub-areas of the geographic area that are between the roads of thegeographic area. The map data 102 of FIG. 1 may be an example of theobtained map data, including the road data and the general area data.The general area data 105 of FIG. 1 may be an example of the obtainedroad data. The analysis module 107 of FIG. 1 may be an example of theobtained general area data.

In some embodiments, autonomous-vehicle data that relates to sensorinformation used by autonomous-vehicle software of an autonomous vehicleto navigate roadways may be obtained. In some embodiments, theautonomous-vehicle data may include sensor information that is receivedby the autonomous-vehicle software. Additionally or alternatively, theautonomous-vehicle data may include sensor information that is receivedby the autonomous-vehicle software and then used by theautonomous-vehicle software (e.g., to navigate roadways).

In these or other embodiments, one or more relationships between theroad data and the sensor information used by the autonomous-vehiclesoftware to navigate roadways may be obtained. Additionally oralternatively, one or more relationships between the sensor informationused by the autonomous-vehicle software and the general area data may beobtained. The relationships may relate the autonomous-vehicle data tothe map data. For example, the relationships may describe howinformation derived from the map data (e.g., simulated sensor readingsbased on 3-D environmental representations representing objects in themap data) may be used by autonomous-vehicle software to navigateroadways.

In these or other embodiments, one or filters, which may include rulesfor filtering map data, may be obtained. The filters may be based on theone or more relationships, and/or the autonomous-vehicle data.

At block 630, filtered road data may be obtained from the map data byselecting (which may include filtering) the high definition road dataaccording to a first degree of accuracy. The first degree of accuracymay be based on a first relationship between the road data and sensorinformation used by autonomous-vehicle software of an autonomous vehicleto navigate roadways. The filtered road data 113 of FIG. 1 may, forexample, include the obtained filtered road data. In some embodiments,the filtering module 110 of FIG. 1 may select the map data.

At block 640, filtered general area data according to a second degree ofaccuracy may be obtained. The filtered general area data may be obtainedby selecting (which may include filtering) the general area dataaccording to the second degree of accuracy. The second degree ofaccuracy may be less accurate than the first degree of accuracy. Thesecond degree of accuracy may be based on a second relationship betweenthe sensor information used by the autonomous-vehicle software and thegeneral area data. The filtered general area data 115 of FIG. 1 may, forexample, include the obtained filtered general area data. In someembodiments, the filtering module 110 of FIG. 1 may select the generalarea data.

At block 650, a three-dimensional environmental representation thatrepresents the geographic area may be generated based on the filteredroad data and the filtered general area data. The three-dimensionalenvironmental representation may be configured as a simulationenvironment with respect to testing the autonomous-vehicle software. The3-D environmental representation 122 of FIG. 1 may be an example of thegenerated three-dimensional environmental representation. Theenvironmental representation-creation module 120 of FIG. 1 may generatethe three-dimensional environmental representation.

In some embodiments, the map data may, additionally or alternatively,include terrain data that indicates elevation information of thegeographical area. In these or other embodiments, the method 600 mayinclude obtaining filtered terrain data by selecting (which may includefiltering) the terrain data according to a third degree of accuracy. Thethird degree of accuracy may be based on the filtering of the road dataaccording to the first degree of accuracy. For example, the third degreeof accuracy may be the same as, or no greater than, the first degree ofaccuracy. As another example, the third degree of accuracy may be basedon an error margin with respect to the filtering of the road dataaccording to the first degree of accuracy. In these or otherembodiments, the method 600 may include generating the three-dimensionalenvironmental representation based on the filtered terrain data.

The method 600 may improve the efficiency and efficacy of generating 3-Denvironmental representations and/or 3-D models including 3-Denvironments for autonomous-vehicle software testing. For example, themethod 600 may remove extraneous map data such that 3-D environmentalrepresentations generated based on the filtered map data may not includeextraneous data.

Modifications, additions, or omissions may be made to the method 600without departing from the scope of the present disclosure. For example,the operations of method 600 may be implemented in differing order.Additionally or alternatively, two or more operations may be performedat the same time. Furthermore, the outlined operations and actions areonly provided as examples, and some of the operations and actions may beoptional, combined into fewer operations and actions, or expanded intoadditional operations and actions without detracting from the essence ofthe disclosed embodiments.

FIG. 7 is a flowchart of an example method 700 of generating asimulation environment, according to at least one embodiment describedin the present disclosure. The method 700 may be performed by anysuitable system, apparatus, or device with respect to the simulationenvironment. For example, one or more of the filtering module 110 ofFIG. 1 , the filtering module 222 of FIG. 2 , the environmentalrepresentation-creation module 120 of FIG. 1 , the mesh generationmodule 250 of FIG. 2 , or the computing system 500 of FIG. 5 (e.g., asdirected by one or more modules) may perform one or more of theoperations associated with the method 700. Although illustrated withdiscrete blocks, the steps and operations associated with one or more ofthe blocks of the method 700 may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the particularimplementation.

The method 700 may begin at block 705, where high definition (HD)mapping data may be obtained. The HD mapping data may include a pointcloud. The HD mapping data may include multiple data topics. Each datatopic may correspond to different types of elements in a geography. Eachdata topic may include one or more distinct instances and each distinctinstance may correspond with a particular object of the data type.

At block 710, multiple meshes may be created. Each mesh may be createdfrom a corresponding distinct instance and may include multiple polygonsformed from multiple points in the point cloud of the HD mapping data.At block 715, multiple textures may be defined for each data topic ofthe multiple data topics.

At block 720, a first data topic of the multiple data topics may beselected. In some embodiments, the first data topic may be a lane linedata topic. At block 725, a first distinct instance of the first datatopic may be selected.

At block 730, a first texture of the multiple textures for the firstdata topic may be selected. In some embodiments, when the first datatopic is a lane line data topic, the multiple textures may include cleanlines, cracked lines, and faded lines. At block 735, the first texturemay be applied to a first mesh corresponding with the first distinctinstance.

At block 740, a second distinct instance of the first data topic may beselected. At block 745, a second texture of the multiple textures forthe first data topic may be selected. The second texture may bedifferent from the first texture.

At block 750, the second texture may be applied to a second meshcorresponding with the second distinct instance. At block 755, the firstmesh and the first texture may be combined with the second mesh and thesecond texture to generate a combined mesh. In some embodiments, thecombined mesh may be included in or used with a 3-D environmentalrepresentation such as described above.

The method 700 may improve the efficiency and efficacy of generating 3-Denvironmental representations and/or 3-D models including 3-Denvironments for autonomous-vehicle software testing. For example, themethod 700 may allow more efficient creation of testing environments andmay allow more robust testing by providing multiple distinct texturesfor particular elements in an environment.

Modifications, additions, or omissions may be made to the method 700without departing from the scope of the present disclosure. For example,the operations of method 700 may be implemented in differing order.Additionally or alternatively, two or more operations may be performedat the same time. Furthermore, the outlined operations and actions areonly provided as examples, and some of the operations and actions may beoptional, combined into fewer operations and actions, or expanded intoadditional operations and actions without detracting from the essence ofthe disclosed embodiments. For example, in some embodiments, the method700 may also include rendering a simulation environment using thecombined mesh. The simulation environment may include one or moreautonomous vehicles. Alternatively or additionally, in some embodiments,the method 700 may include adding a decal as an additional element tothe first texture.

Alternatively or additionally, in some embodiments, the method 700 mayinclude obtaining database data, the database data including arepresentation of one or more buildings as a building layer. The methodmay also include obtaining scenario data, the scenario data including arepresentation of one or more trees as a static object layer. The methodmay also include combining the building layer, the static object layer,and the combined mesh to generate a 3-D scenario environment.

Alternatively or additionally, in some embodiments, the method 700 mayinclude selecting a second data topic of the plurality of data topics.The method may also include selecting a third distinct instance of thesecond data topic. The method may further include selecting a thirdtexture of the plurality of textures for the second data topic. Themethod may also include applying the third texture to a third meshcorresponding with the third distinct instance. In these and otherembodiments, the first mesh and the first texture may be combined withthe second mesh and the second texture and the third mesh and the thirdtexture to generate the combined mesh.

Alternatively or additionally, in some embodiments, the method 700 mayfurther include dividing the HD mapping data into separate groups bydata topic. The method may also include separately processing each groupto filter the HD mapping data. The method may further include separatelystoring each processed group.

As indicated above, the embodiments described in the present disclosuremay include the use of a special purpose or general purpose computer(e.g., the processor 502 of FIG. 5 ) including various computer hardwareor software modules, as discussed in greater detail below. Further, asindicated above, embodiments described in the present disclosure may beimplemented using computer-readable media (e.g., the memory 504 or datastorage 506 of FIG. 5 ) for carrying or having computer-executableinstructions or data structures stored thereon.

As used in the present disclosure, the terms “module” or “component” mayrefer to specific hardware implementations configured to perform theactions of the module or component and/or software objects or softwareroutines that may be stored on and/or executed by general purposehardware (e.g., computer-readable media, processing devices, etc.) ofthe computing system. In some embodiments, the different components,modules, engines, and services described in the present disclosure maybe implemented as objects or processes that execute on the computingsystem (e.g., as separate threads). While some of the system and methodsdescribed in the present disclosure are generally described as beingimplemented in software (stored on and/or executed by general purposehardware), specific hardware implementations or a combination ofsoftware and specific hardware implementations are also possible andcontemplated. In this description, a “computing entity” may be anycomputing system as previously defined in the present disclosure, or anymodule or combination of modulates running on a computing system.

Terms used in the present disclosure and especially in the appendedclaims (e.g., bodies of the appended claims) are generally intended as“open” terms (e.g., the term “including” should be interpreted as“including, but not limited to,” the term “having” should be interpretedas “having at least,” the term “includes” should be interpreted as“includes, but is not limited to,” etc.).

Additionally, if a specific number of an introduced claim recitation isintended, such an intent will be explicitly recited in the claim, and inthe absence of such recitation no such intent is present. For example,as an aid to understanding, the following appended claims may containusage of the introductory phrases “at least one” and “one or more” tointroduce claim recitations. However, the use of such phrases should notbe construed to imply that the introduction of a claim recitation by theindefinite articles “a” or “an” limits any particular claim containingsuch introduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitationis explicitly recited, those skilled in the art will recognize that suchrecitation should be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, means at least two recitations, or two or more recitations).Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” isused, in general such a construction is intended to include A alone, Balone, C alone, A and B together, A and C together, B and C together, orA, B, and C together, etc.

Further, any disjunctive word or phrase presenting two or morealternative terms, whether in the description, claims, or drawings,should be understood to contemplate the possibilities of including oneof the terms, either of the terms, or both terms. For example, thephrase “A or B” should be understood to include the possibilities of “A”or “B” or “A and B.”

All examples and conditional language recited in the present disclosureare intended for pedagogical objects to aid the reader in understandingthe present disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions. Althoughembodiments of the present disclosure have been described in detail,various changes, substitutions, and alterations could be made heretowithout departing from the spirit and scope of the present disclosure.

1. (canceled)
 2. A computer-implemented method, comprising: accessingmap data of a region, the map data of the region including at least oneof: image data, vehicle sensor data, or global positioning data;separating the map data into a plurality of data topics; and storing theplurality of data topics in a data storage, the data storage beingaccessible by one or more simulation environment generation systems, theone or more simulation environment generation systems capable to accessthe plurality of data topics in the data storage to generate asimulation environment.
 3. The computer-implemented method of claim 2,further comprising defining a respective texture for each data topic inthe plurality of data topics.
 4. The computer-implemented method ofclaim 2, further comprising defining a second texture for a first datatopic in the plurality of data topics such that the first data topic hasat least two defined textures, including generating instructions for howto apply the textures to data topics.
 5. The computer-implemented methodof claim 4, further comprising defining a plurality of texturesavailable to a second data topic, wherein the simulation environmentincludes multiple instances of the second data topic, wherein whenrendering the simulation environment, a decision is made to select atleast one texture for each instance of the second data topic.
 6. Thecomputer-implemented method of claim 2, wherein the image data includesinformation obtained by a vehicle that traveled in the region andcaptured images of the region.
 7. The computer-implemented method ofclaim 2, wherein the vehicle sensor data includes sensor informationcaptured by one or more sensors of one or more vehicles in the region.8. The computer-implemented method of claim 2, wherein the vehiclesensor data includes data in a point-cloud format.
 9. Thecomputer-implemented method of claim 2, wherein the map data is capturedsubstantially in real-time by a vehicle travelling in the region. 10.The computer-implemented method of claim 2, wherein separating the mapdata into the plurality of data topics included creating at least onenested data topic.
 11. The computer-implemented method of claim 10,wherein the at least one nested data topic includes a top-level datatopic and at least one subtopic, wherein each subtopic of the at leastone subtopic is to define at least one variation of the top-level datatopic.
 12. The computer-implemented method of claim 2, wherein each datatopic of the plurality of data topics is stored as a separate file inthe data storage.
 13. The computer-implemented method of claim 21,wherein the simulation environment includes a plurality of layers. 14.The computer-implemented method of claim 13, wherein the plurality oflayers include a first layer for the image data, a second layer for thevehicle sensor data, and a third layer for the global positioning data.15. The computer-implemented method of claim 14, the vehicle sensor dataincluding a first vehicle sensor data type and a second vehicle sensordata type, the second layer being for the first vehicle sensor datatype, the plurality of layers including a fourth layer for the secondvehicle sensor data type.
 16. The computer-implemented method of claim2, wherein the map data includes at least one of terrain data and roaddata.
 17. The computer-implemented method of claim 2, wherein a firstdata topic of the plurality of data topics is generated by grouping aplurality of points within a point cloud based on intensity informationof the plurality of points within the point cloud.
 18. Acomputer-implemented method, comprising: accessing a data storage fordata topics; identifying, within the data storage, one or more datatopics for inclusion in a simulation environment; identifying a firsttexture for the data topic and a second texture for the data topic,including instructions for applying the first texture for the data topicand the second texture for the data topic in the simulation environment;generating the simulation environment with the data topic includingapplying the first texture to the data topic and the second texture tothe data topic in the simulation environment according to theinstructions.
 19. The computer-implemented method of claim 18, furthercomprising supplementing the simulation environment with an object thatis not included in a set of map data that was used when creating thedata topic.
 20. The computer-implemented method of claim 18, whereingenerating the simulation environment includes generating a mesh basedon two or more data topics.
 21. The computer-implemented method of claim18, wherein the first texture and the second texture are representativeof different real-world examples of the data topic.